Method for ranking analysis tools

ABSTRACT

Analysis tools are used for resolving a service request for software performance problems. Ranking of the analysis tools includes measuring a plurality of times to resolution of a plurality of service requests for software performance problems after runnings of a plurality of analysis tools are initiated; capturing sets of errors in the plurality of service requests; storing identities of the plurality of analysis tools with the times to resolution of the service requests and the sets of errors; determining an average time to resolution of each of the plurality of analysis tools for each set of errors; organizing the plurality of analysis tools into one or more categories using the sets of errors; and ranking the analysis tools within each category using the average times to resolution of the analysis tools within the category.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of co-pending U.S. patentapplication Ser. No. 13/413,993, filed on Mar. 7, 2012.

BACKGROUND

Customers of software products often contact vendor support personnelwhen software problems occur. For example, as part of the supportprocess, a customer may electronically submit a service request tosupport personnel. The service request includes a collection of filesthat contain the errors at issue. When a support person begins workingon a service request, she must choose from a variety of differentanalysis tools to run on the service request files. Selecting the mostappropriate or efficient analysis tool for a particular error or sets oferrors can sometimes be difficult. Support personnel often rely upontheir own experience or the recommendation of those with more technicalexpertise. Some attempt to highlight certain analysis tools by rankingthe analysis tools by popularity or alphabetically. However, theseapproaches do not necessarily highlight the most appropriate orefficient analysis tools for the customer's specific problem.

SUMMARY

According to one embodiment of the present invention, in a method forranking analysis tools by a computing system, a processor of thecomputing system measures a time to resolution of a given softwareperformance problem for each run of any of a plurality of analysistools. The processor determines an average time to resolution of thegiven software performance problem for each of the plurality of analysistools and ranks the plurality of analysis tools using the average timesto resolution of the given software performance problems for theplurality of analysis tools.

In one aspect of the present invention, measuring the time to resolutionof the given software performance problem for each run of any of theplurality of analysis tools comprises receiving a service request forthe software performance problem, wherein the service request includesone or more files, and includes initiating a running of a given analysistool of the plurality of analysis tools. A time to resolution of theservice request is measured from a predetermined time after the givenanalysis tool was initiated. A set of errors in the files of the servicerequest is captured and an identity of the given analysis tool is storedalong with the time to resolution of the service request and the set oferrors.

In one aspect of the present invention, determining the average time toresolution of the given software performance problem for each of theplurality of analysis tools comprises aggregating the stored identity ofthe given analysis tool with the time to resolution of the servicerequest and the set of errors across a plurality of service requests anddetermining an average time to resolution of each of the plurality ofanalysis tools for each set of errors.

In one aspect of the present invention, the ranking the plurality ofanalysis tools using the average times to resolution of the givensoftware performance problems for the plurality of analysis toolscomprises organizing the plurality of analysis tools into one or morecategories using the sets of errors aggregated across the plurality ofservice requests. Analysis tools are ranked within each category usingthe average times to resolution of the analysis tools within thecategory.

In one aspect of the present invention, receiving a second servicerequest for a second software performance problem, wherein the secondrequest includes one or more files. A second set of errors is capturedin the one or more second service request files and a given category ofanalysis tools is determined using the second set of errors. One or moreanalysis tools in the given category is recommended based on therankings of the analysis tools within the given category.

System and computer program products corresponding to theabove-summarized methods are also described and claimed herein.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 illustrates an embodiment of a system for ranking analysis toolsaccording to the present invention.

FIG. 2 is a flowchart illustrating an embodiment of a method for rankinganalysis tools according to the present invention.

FIG. 3 is a flowchart illustrating in further detail the embodiment ofthe method for ranking analysis tools according to the presentinvention.

FIG. 4 is a flowchart illustrating an embodiment of a method forrecommending analysis tools according to the present invention.

DETAILED DESCRIPTION

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java® (Java, and all Java-based trademarks and logos aretrademarks of Sun Microsystems, Inc. in the United States, othercountries, or both), Smalltalk, C++ or the like and conventionalprocedural programming languages, such as the “C” programming languageor similar programming languages. The program code may execute entirelyon the user's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer special purposecomputer or other programmable data processing apparatus to produce amachine, such that the instructions, which execute via the processor ofthe computer or other programmable data processing apparatus, createmeans for implementing the functions/acts specified in the flowchartand/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified local function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theembodiment was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

FIG. 1 illustrates an embodiment of a system for ranking analysis toolsaccording to the present invention. The computer system 100 isoperationally coupled to a processor or processing units 106, a memory101, and a bus 109 that couples various system components, including thememory 101 to the processor 106. The bus 109 represents one or more ofany of several types of bus structures, including a memory bus or memorycontroller, a peripheral bus, an accelerated graphics port, and aprocessor or local bus using any of a variety of bus architectures. Thememory 101 may include computer readable media in the form of volatilememory, such as random access memory (RAM) 102 or cache memory 103, ornon-volatile storage media 104. The memory 101 may include at least oneprogram product having a set of at least one program code modules 105that are configured to carry out the functions of embodiments of thepresent invention when executed by the processor 106. The computersystem 100 may also communicate with one or more external devices 111,such as a display 24, via I/O interfaces 107. The computer system 100may communicate with one or more networks via network adapter 108.

FIG. 2 is a flowchart illustrating an embodiment of a method performedby a circuit, module or system for ranking analysis tools according tothe present invention. The method measures the time to resolution of agiven software performance problem for each run of any of a plurality ofanalysis tool available to a support person (201). Using this data, themethod determines an average time to resolution of the given performanceproblem for each of the plurality of analysis tools (202). The methodthen ranks the plurality of analysis tools using the average times toresolution of the given software performance problem (203).Recommendations of analysis tools may be made to support personnel forresolving the given software performance problem based on the rankings.In this manner, the embodiments of the present invention assist in theselection of the most appropriate or efficient analysis tools forresolving particular software problems.

FIG. 3 is a flowchart illustrating in further detail the embodiment ofthe method for ranking analysis tools according to the presentinvention. Customers submit service requests to support personnel forassistance with a software performance problem. The service requestcontains files, such as emails, system logs, etc. Contained within theservice request files are errors that may identity the problems, i.e.,serve as “fingerprints” or “symptoms” of the problem. In response toreceiving a service request comprising the files from a customer (301),the circuit, module or system receives a signal from the support personinitiating the running of an analysis tool of a plurality of analysistools, and the method measures the time to resolution of the servicerequest from a predetermined time after the analysis tool is initiated(302). For example, when the analysis tool is initiated, the methodstarts a timer for the service request. The timer will run until theservice request is resolved. Further, the method examines the servicerequest files and captures a set of errors in the service request files(303). The method then stores the identity of the analysis tool that wasrun along with the time to resolution and the set of errors (304). Steps301 through 304 are repeated each time an analysis tool is run.

The method aggregates the data collected across multiple servicerequests (305), and uses the aggregated data to determine the averagetime to resolution of each analysis tool for each set of errors (306).The method further uses the aggregated data to organize the analysistools into categories using the sets of errors (307). For example, whena memory analyzer tool and a heap analyzer tool are most often run onthe service requests containing files with Out of Memory exceptions andNull Pointer Exceptions, then the method can place the memory analyzerand heap analyzer tools into the category for Out of Memory and NullPointer Exceptions. The analysis tools within each category are thenranked using their respective average times to resolutions (308). Forexample, when the average time to resolution for the memory analyzertool is one hour and the average time to resolution for the heapanalyzer tool is two hours, the method can rank the memory analyzer toolhigher than the heap analyzer tool.

The rankings may then be used to provide recommendations of whichanalysis tools for use with which sets of errors. FIG. 4 is a flowchartillustrating an embodiment of a method for recommending analysis toolsaccording to the present invention. When a support person receives aservice request comprising files from a customer (401), the methodcaptures the set of errors found in the service request files (402). Themethod then determines the category of analysis tools using the set oferrors (403). The method then can recommend one or more analysis toolsbased on their rankings within that category (404). For example, assumethat the service request files from the customer contain Out of Memoryand Null Pointer Exceptions. The method captures these errors for theservice request files and determines that they are relevant to the Outof Memory and Null Pointer Exceptions category. The method can thenrecommend the memory analyzer tool over the heap analyzer tool since thememory analyzer tool is ranked higher than the heap analyzer tool inthis category.

Considerations other than or in addition to the average time toresolution may be used to rank the analysis tools. For example, thenumber of clicks that occur in a service request after an analysis toolis run can be recorded. This information may indicate the amount ofinteraction that an analysis tool may require after it is initiallyinvoked to resolve a service request. Other possible considerations mayinclude: time that the analysis tool is displayed in foreground or thebackground, which may indicate the amount of actual interaction with theanalysis tool by the support person; or the amount of emailcommunications between the support person and the customer after ananalysis tool is initiated, which may indicate the level of complexityof the problem. These considerations may be used to increase theaccuracy of the rankings and thus the recommendations based on therankings.

Further, embodiments of the method may recommend a chain of analysistools to use for a particular set of errors. When a first recommendedanalysis tool does not resolve the service request, the method mayrecommend the next higher ranking analysis tool, and so forth. Theembodiments of the method may further recommend particular combinationsof analysis tools to use for a particular set of errors, where thecombination is determined to lead to faster time to resolution.

The descriptions of the various embodiments of the present invention hasbeen presented for purposes of illustration, but are not intended to beexhaustive or limited to the embodiments disclosed. Many modificationsand variations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

What is claimed is:
 1. A method for ranking analysis tools by acomputing system comprising a processor, comprising: measuring by theprocessor a time to resolution of a given software performance problemfor each run of any of a plurality of analysis tools; determining, bythe processor, an average time to resolution of the given softwareperformance problem for each of the plurality of analysis tools; andranking, by the processor, the plurality of analysis tools using theaverage times to resolution of the given software performance problemsfor the plurality of analysis tools.
 2. The method of claim 1, whereinthe measuring the time to resolution of the given software performanceproblem for each run of any of the plurality of analysis toolscomprises: receiving a service request for the software performanceproblem, wherein the service request includes one or more files;initiating a running of a given analysis tool of the plurality ofanalysis tools; measuring a time to resolution of the service requestfrom a predetermined time after the given analysis tool is initiated;capturing a set of errors in the one or more files of the servicerequest; and storing an identity of the given analysis tool with thetime to resolution of the service request and the set of errors.
 3. Themethod of claim 2, wherein the determining the average time toresolution of the given software performance problem for each of theplurality of analysis tools comprises: aggregating the stored identityof the given analysis tool with the time to resolution of the servicerequest and the set of errors across a plurality of service requests;and determining an average time to resolution of each of the pluralityof analysis tools for each set of errors.
 4. The method of claim 3,wherein the ranking the plurality of analysis tools using the averagetimes to resolution of the given software performance problems for theplurality of analysis tools comprises: organizing the plurality ofanalysis tools into one or more categories using the sets of errorsaggregated across the plurality of service requests; and rankinganalysis tools within each category using the average times toresolution of the analysis tools within the category.
 5. The method ofclaim 4, further comprising: receiving a second service request for asecond software performance problem, wherein the second service requestincludes one or more files; capturing a second set of errors in thesecond service request files; determining a given category of analysistools using the second set of errors; and recommending one or moreanalysis tools in the given category based on the rankings of theanalysis tools within the given category.
 6. A method for rankinganalysis tools by a computing system comprising a processor, comprising:measuring, by the processor, a plurality of times to resolution of aplurality of service requests for software performance problems afterrunnings of a plurality of analysis tools are initiated; capturing, bythe processor, sets of errors in the plurality of service requests;storing, by the processor, identities of the plurality of analysis toolswith the times to resolution of the service requests and the sets oferrors; determining, by the processor, an average time to resolution ofeach of the plurality of analysis tools for each set of errors;organizing, by the processor, the plurality of analysis tools into oneor more categories using the sets of errors; and ranking, by theprocessor, the analysis tools within each category using the averagetimes to resolution of the analysis tools within the category.
 7. Themethod of claim 6, further comprising: receiving a given service requestfor a given software performance problem, wherein the given servicerequest includes one or more files; capturing a given set of errors inthe given service request files; determining a given category ofanalysis tools using the given set of errors; and recommending one ormore analysis tools in the given category based on the rankings of theanalysis tools within the given category.